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Abstract 

The study investigated whether online professional development courses with different levels of 
support have different impacts on teacher outcomes. Variations of an online course for middle 
school algebra teachers were created for four experimental conditions. One was a highly sup- 
ported condition, with a math education instructor, an online facilitator, and asynchronous 
peer interactions among participants available as participants worked through the course to- 
gether. Another was a self-paced condition, in which none of the supports were available. The 
other two conditions provided intermediate levels of support. All conditions showed significant 
impact on teachers’ mathematical understanding, pedagogical beliefs, and instmctional prac- 
tices. Surprisingly, the positive outcomes were comparable across all four conditions. Further 
research is needed to determine whether this finding is limited to self-selected teachers, the spe- 
cifics of this online course, or other factors that limit generalizability. (Keywords: Online pro- 
fessional development, distance learning, e-learning, online facilitation, self-paced learning) 

INTRODUCTION 

Increasing demand for teacher professional development and significant 
advances in information and communication technologies have led to a 
proliferation of online professional development programs (OPD) in recent 
years (Dede, 2006; Galley, 2002). Online courses and workshops have become 
common in both preservice and inservice teacher education programs, using 
a variety of technologies to provide learning opportunities for educators that 
differ from traditional, face-to-face courses and workshops in many ways. First 
and foremost, online learning can be used to bridge distance and time: Educa- 
tors can participate via the Internet or other technologies rather than travel to 
a specific site, and interactions can be asynchronous so that participants do not 
need to be available at the same time. Online learning can thereby broaden 
the professional learning opportunities available to educators. In addition to 
its logistical advantages, OPD can provide expertise and resources to locations 
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where they would not be available otherwise; provide new means to interact 
with experts and colleagues; enable educators to experience for themselves new 
forms of teaching and learning; and make participation in coaching, mentoring, 
and professional learning communities more accessible (Kleiman, 2004). 

Online professional learning can be provided in many forms. For example, 
distance learning classrooms enable individuals to participate in a class via video 
conferencing, with the goal of making the online experience as close as possible 
to an in-class experience. As this approach requires simultaneous participation, 
it bridges distance but not time. Alternatively, an online course can be con- 
ducted completely through asynchronous interactions, so that each participant 
can work with the course resources and add their own responses and questions 
to the course discussion at any time. Technologies can range from multipoint 
video conferencing to one-way transmission of Web-based resources, but they 
often involve synchronous and / or asynchronous voice and text exchanges 
among participants and instructors. Another variation is self-paced online 
courses, in which each participant works through a series of resources and 
activities at his or her own pace, often with an instructor available to respond to 
questions and provide feedback about submitted assignments. 

To be effective, OPD for teachers must reflect the principles of effective 
professional development. Research has led to agreement on a number of key 
principles of successful professional development practices for K-12 educators. 
Major studies and syntheses by Stigler and Stevenson (1991), Darling-Ham- 
mond and McLaughlin (1995), Sparks and Hirsch (1997), Ball and Cohen 
(1999), National Foundation for the Improvement of Education (1996), Borasi 
and Fonzi (2002), and others consistently agree that professional development 
is more effective when it: 

• Fosters a deepening of subject-matter knowledge, a greater understanding 
of learning, and a greater appreciation of students’ needs 

• Centers around the critical activities of teaching and learning — planning 
lessons, evaluating student work, developing curriculum, improving class- 
room practices, and increasing student learning — rather than on abstrac- 
tions and generalities 

• Builds on investigations of practice through cases that involve specific 
problems of practice questions, analysis, reflection, and substantial profes- 
sional discourse 

• Values and cultivates a culture of collegiality, involving knowledge and 
experience sharing among educators; and, 

• Is sustained, intensive, and continuously woven into the everyday fabric of 
the teaching profession through modeling, coaching, and collaborations. 

Although there is general consensus about the key principles of effective 
teacher professional development, in actual practice, factors such as required ex- 
pertise, funding, access from rural areas, or opportunities for collaboration may 
inhibit implementation. OPD can incorporate characteristics associated with ef- 
fective face-to-face professional development and offers a number of advantages 
over face-to-face formats. Because Internet access has become nearly universal in 
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K-12 schools and widespread in teachers’ homes, OPD provides teachers with 
opportunities to engage in forms of training that may not be available within 
their local areas. Teachers can participate in professional development during 
times that are convenient, receive job-embedded support that addresses im- 
mediate classroom needs, customize programs to better suit their own indi- 
vidual learning styles, interact with material through a variety of visual or other 
multimedia formats, and gain valuable computer and online technology skills 
(Docherty & Sandhu, 2006; Garrison & Cleveland-Innes, 2005; Ginsburg, 
Gray, & Levin, 2004; National Staff Development Council, 2001; Richardson, 
2002; Spicer, 2002; Treacy, Kleiman, & Peterson, 2002). 

Due to its accessible and flexible modes of communication, OPD can also 
support the growth and maintenance of teacher learning communities. The 
Internet can connect teachers to broad networks of professionals, provide them 
with access to a wide array of learning experiences, and help to reduce the 
isolation that has often accompanied traditional forms of classroom teaching 
(DuFour, 2002; Ginsburg, Gray, & Levin, 2004; National Staff Development 
Council, 2001). Some researchers and educators argue that OPD can promote 
deeper levels of communication and thought among teachers than face-to-face 
forms of professional development. Because online programs can store written 
records of teacher conversations, and because teachers can participate in group 
discussion asynchronously, OPD allows teachers to contribute ideas when they 
are ready and to be more reflective in their written online comments (Spicer, 
2002; Treacy, Kleiman, & Peterson, 2002). 

Although researchers have reached a broad consensus about the general com- 
ponents of high-quality K-12 teacher professional development in face-to-face 
contexts (Sparks, 2002), many questions remain about the design and delivery 
of effective OPD. Recent studies have focused on describing the characteris- 
tics of existing OPD programs, but little research has examined specific OPD 
formats and their impacts on teacher practice and student learning (Ginsburg, 
Gray, & Levin, 2004; Lawless & Pellegrino, 2007; Whitehouse, Breit, McClo- 
skey, Ketelhut, & Dede, 2006). In a strategic analysis of OPD for mathematics 
teachers, Ginsburg, Gray, and Levin (2004) reviewed the quality and effective- 
ness of more than 40 OPD programs, judging them against what is known 
about similar evidence for effectiveness and quality in traditional face-to-face 
professional development. Overall, Ginsburg et al. found that the OPD sites 
lacked rigorous independent assessments of the impact of OPD on teacher or 
student outcomes. Similarly, in a review of the literature on technology profes- 
sional development for teachers, Lawless and Pellegrino (2007) indicate that the 
success of professional development is typically judged based on anecdotal evi- 
dence or the results of surveys in which teachers indicate their satisfaction with 
the experience or their perception of its usefulness to their work. Consequently, 
we know only what teachers think about their professional development activi- 
ties, not what they actually learn, how it changes their pedagogies, or how it 
impacts student learning (Lawless & Pellegrino, 2007). 
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Key Components of Online Professional Development 

The role of facilitators and content experts in online courses has been the top- 
ic of many OPD design recommendations and some research. Researchers and 
practitioners such as Anderson Rourke, Garrison, and Archer (2001), Treacy et 
al. (2002), and Yang and Liu (2004) suggest a number of functions that facilita- 
tors should serve to enhance the quality of OPD programs. For example, they 
should investigate participants’ needs, require regular contributions to online 
discussions, and provide clear structure for participant dialogue to promote 
active inquiry and thoughtful online discourse. They argue that facilitators need 
to pose engaging questions, challenge participants’ positions, identify areas 
of conflict, and help participants make connections between different ideas. 
Some researchers have found that OPD facilitators cannot simply ask teachers 
to describe how they teach and to “be specific” in their online postings when 
examining issues surrounding classroom instruction; more meaningful discourse 
arises only when facilitators request participants to analyze, compare, and reflect 
upon concrete cases (Nemirovsky & Galvis, 2004). Online facilitators also play 
an important role in eliciting the contributions of less active participants, man- 
aging participants who may dominate discussions, and creating a comfortable 
and social environment (Anderson, et al., 2001; Florida Instructional Technol- 
ogy Resource Center at UCF, 2000; Sargeant, Curran, Allen, Jarvis-Selinger, & 
Ho, 2006; Treacy, Kleinian, & Peterson, 2002). 

In certain subject areas, OPD may be most effective when a content expert 
acts as a facilitator (Anderson, et al., 2001). Content experts can help diagnose 
and correct teacher misconceptions, recommend useful and reliable resources, 
validate new insights, and provide context for new learning. Furthermore, the 
presence of a knowledgeable online instructor can help participants synthesize 
course material and progress beyond the initial stages of idea discovery and 
exploration (Garrison & Cleveland-Innes, 2005). 

Although the recommendations that researchers and practitioners have issued 
may be helpful to those currently designing and implementing OPD pro- 
grams, as previously noted, most existing recommendations lack a base in solid 
experimental research. Consequently, the true impacts of OPD content expert 
instructors, facilitators, and participant interaction remain unclear. To help 
build knowledge about the effectiveness of different OPD models on teacher 
learning, the study presented in this article conducted a large-scale experiment 
to examine the following question: Do online professional development models 
with different types of facilitation, levels of interaction among participants, and 
pacing schedules have different impacts on teacher outcomes? 

The research addresses whether online courses for educators should be 
designed to optimize schedule flexibility for participants, to optimize interac- 
tions between each individual participant and the instructors, or to optimize 
interactions among a cohort of peers participating in the course together. The 
decision about what to optimize is fundamental to both designing and imple- 
menting online professional learning programs. Drawing upon prior research 
observations and recommendations, we hypothesized that OPD courses will 
have greater impact on teachers’ knowledge and practice when interactions with 
content experts, online facilitators, and peers are available. 
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The research reported in this article was one of a series of studies conducted 
to provide a research basis to inform these types of decisions. A prior study by 
the same research team (Carey, Kleiman, Russell, & Douglas, 2008) compared 
the learning outcomes from two designs of online workshops for teachers of 
middle school mathematics. One was a self-paced course that participants com- 
pleted on their own schedules without interacting with peers and with minimal 
interaction with instructors. Participants in this condition could complete the 
10-session workshop on their own schedules, with the only time constraint be- 
ing the completion of all the required assignments in a maximum of 10 weeks. 
The second type of design was a cohort-based course in which participants 
worked through the material following a structured timeline and frequent inter- 
actions with both instructors and peer participants were integral to the design 
of each session. The interactions were asynchronous and text-based, so that each 
participant could complete the session assignments and contribute to the dis- 
cussion whenever it was convenient during the 1-week window for each session. 
The course goals and content were kept as constant as possible across the two 
designs. Although we hypothesized that the cohort-based course would have 
a larger effect on the intended learning outcomes, the study found that both 
versions of the course were equally effective at creating the desired changes in 
teachers’ pedagogical beliefs, knowledge of teaching mathematics, and instruc- 
tional practices. 

The study presented here sought to replicate and extend this prior unexpected 
finding by comparing the effects of four variations of an online course, again for 
middle school mathematics teachers. We designed these variations to enable us 
to investigate the value of various types of interactions within an online course 
for adult professional learners, ranging from a highly interactive cohort model 
facilitated by content and online interaction experts to a fully self-paced course 
with minimal interactions. 

In the sections that follow, we describe the course and levels of interactivity 
that were varied across the four conditions in the study. We then describe the 
methodology employed to recruit and assign teachers to groups, as well as the 
data-collection instruments employed for this study. Finally, we present the 
findings from the study and explore the implications these findings may have 
for the design and delivery of online professional development. 

Online Course: Building Algebraic Thinking in the Middle Grades 

The online course employed for this study was titled Building Algebraic 
Thinking in the Middle Grades and was based on a book authored by Mark 
Driscoll titled Fostering Algebraic Thinking (Driscoll, 1999). The course was de- 
veloped by mathematics experts at Education Development Center Inc. (EDC) 
and focused on the algebraic concepts of patterns and functions. We provided 
four versions of the course to test the impact of different types of interactions. 
As far as possible, the goals, content, and activities were the same across all four 
versions. 

The primary learning goals of the course were to increase teachers’ abilities to 
(a) understand the mathematics of patterns and functions, (b) recognize and 
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Table 1: Course Sessions and Summary 


Session Number and Tide 
Session 1 : Orientation & What 
Is Algebraic Thinking? 


Session 2: Exploring Algebraic 
Thinking in Patterns — 
Examining Your Own Thinking 

Session 3: Analyzing Students’ 
Algebraic Thinking about 
Patterns 

Session 4: Using Teacher 
Questioning to Develop 
Algebraic Thinking 

Session 3: Conducting Your 
Student Interview 


Session 6: Creating Meaning 
with Representations — Graphs 
without Numbers 
Session 7: Creating Meaning 
with Representations — Tables, 
Graphs, Words, and Equations 

Session 8: A Final Look at 
Algebraic Thinking 


Content Summary 

Participants completed the precourse surveys, an orientation 
to the course, and an orientation to the technology needed 
for the assignments. Participants were introduced to various 
definitions of algebraic thinking, including the definition 
primarily used in this course, and were asked to compare 
those definitions with their current understanding. 
Participants focused on their own mathematical thinking 
when doing pattern problems. These activities served as a 
starting point when participants examine student thinking in 
later sessions. 

Participant focus turned to examining students’ algebraic 
thinking to better understand the nature of their misconcep- 
tions and difficulties. 

To conclude the patterns section of this course, participants 
turned to developing good questioning techniques that can 
help students develop algebraic thinking in the context of 
patterns activities. 

This session asked participants to explore how to foster 
students’ algebraic thinking habits as they build generaliza- 
tion skills essential to understanding functions. Participants 
conducted two student interviews to gain insight into their 
algebraic thinking related to functions. 

This session explores ways participants could foster students’ 
algebraic thinking habits as they learn to analyze and inter- 
pret graphs of functions. 

This session continues the work with algebraic thinking 
related to functions and addresses ways to help students 
make sense of the relationship between different function 
representations. 

Participants complete and submit their final project and 
complete the postcourse surveys. 


build on opportunities with their students for algebraic thinking in a variety of 
kinds of mathematics problems, (c) analyze students’ mathematical thinking 
and identify misconceptions commonly found in students’ work with patterns 
and functions, and (d) use specific instructional methods, especially question- 
asking strategies, that encourage the development of students’ mathematical 
thinking and their abilities to communicate mathematical ideas. 

We divided the course activities into eight sessions, preceded by a brief ori- 
entation pre-session to familiarize participants with the technology and course 
expectations. Sessions contained readings, videos, mathematical activities for 
the teacher participants, and pedagogical activities such as preparing a lesson 
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Table 2: Supports Available in Each of the Four Conditions 


Group 

Mathematics 

Instructor 

Online Facilitator 

Peer interactions 

1 . Highly Supported 

X 

X 

X 

2. Facilitated Peer Support 


X 

X 

3. Instructor Support Only 

4. No Support 

X 

X 



or interviewing students about their mathematics thinking. Depending upon 
the version of the course, session requirements also included participation in an 
asynchronous online discussion or completion of an individual journal entry. 
Participants were expected to spend about 4 hours per session and to complete a 
final project. The course content is summarized in Table 1, and the full course is 
available at http://www.curriki.org/xwiki/bin/view/Coll_edcl/Building 
AlgebraicThinkingThroughPatternFunctionandNumber-Professional 
DevelopmentCourseforMiddleGradeMathTeachers. 

EXPERIMENTAL CONDITIONS 

The four versions of the delivered course varied in the types of supportive 
interactions available to participants. The course versions contained different 
configurations of the following three types of supports: 

1 . A mathematics education content expert who provided detailed feedback 
on assignments, responded to questions about mathematics content and/ 
or pedagogy, and, in courses that also provided an asynchronous discus- 
sion option, contributed both questions to participants and responses to 
participants’ messages in the discussions 

2. An online facilitation expert who was trained to encourage and guide 
participation in online interactions. The online facilitation expert 
welcomed participants to the course, sent twice-weekly reminders to par- 
ticipants about assignment and activities, tracked participation, engaged 
participants in online discussions, and provided technical assistance when 
the need arose. 

3. Interactions among the peer group of teachers participating in the course 
via an asynchronous, text-based discussion board. Participants could view 
and respond to each others responses to discussion prompts contained in 
each session, share ideas, ask each other questions, and respond to ques- 
tions from their peers. 

These three types of supports are referred to in the remainder of this paper as 
(a) mathematics instructor, (b) online facilitator, and (c) peer interactions. 

In the four conditions of this study, the availability of each type of support is 
shown in Table 2. Each course condition is described in more detail below. 

Each session of each version of the course contained preplanned questions. In 
courses with peer interactions, these questions were used to start online discus- 
sions, and the instructor, facilitator, and participants might then post additional 
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related questions according to the flow of the discussion. In the courses without 
peer interactions, each individual participant responded to the preplanned 
questions on their own, with no opportunities for follow-up questions. 

Group 1 received a highly supported course that had both a mathematics 
instructor and online facilitator, both of whom were active in interacting with 
course participants, along with peer interactions. Participants were expected 
to progress through the 8-week sessions together and to engage in weekly 
interactions with other participants, the instructor, and the facilitator through 
an online, asynchronous, text-based discussion board. The instructors posed 
questions each week. Participants responded to the queries, to their colleagues’ 
postings, and to additional questions the instructors posed as they attempted 
to guide the conversation toward deeper understanding of that week’s topic. 

The online facilitators encouraged participants to use the discussion board as 
their primary means of communication with other participants and with the 
mathematics instructors, with a minimum expectation of one initial message 
and two responses to other messages. Participants, instructors, and facilitators 
also communicated with each other via e-mail occasionally to submit the three 
homework assignments, send feedback on the assignments, seek and provide 
technical support, and provide encouragement. On rare occasions, facilita- 
tors and individual participants communicated via telephone. The majority of 
phone interactions took place at the onset of the course, while participants were 
registering and completing the orientation tasks before the course began. 

Group 2 provided participants with facilitated peer support, which involved 
an online facilitator and peer interactions without the mathematics instructor. 
In the absence of an instructor with mathematics education content expertise, 
the facilitator responded to participants’ content questions by directing them to 
the participant discussion board and to other resources. In this condition, the 
facilitator also tracked participation and provided technical assistance. Partici- 
pants were expected to progress through the eight course sessions together and 
to participate in weekly discussions with their peers. Each week, the facilitator 
posted a new question on the discussion board and then interacted as needed. 
She responded to lapses in discussion and to direct questions but, again, did 
not offer direct content instruction. Participants responded to the initial ques- 
tion, to each other’s postings, and to additional queries the facilitator may 
have posed. Although the initial discussion questions for each session were 
preplanned and consistent across the conditions, follow-up questions varied ac- 
cording to the flow of the discussion. 

Although the majority of the interactions occurred via the asynchronous 
discussion board, participants submitted homework assignments via e-mail, and 
the facilitator provided acknowledgement via e-mail to each participant that the 
assignment had been completed. On rare occasions, facilitators and individual 
participants communicated by phone. Again, the majority of phone interac- 
tions took place at the onset of the course while participants were registering 
and completing the orientation tasks. 

Group 3 provided participants with instructor support only. In this condi- 
tion, participants had access to both a mathematics instructor and an online 
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facilitator but did not have a means to interact with the other participants. As 
in Group 1, the instructor with expertise in the teaching of algebra provided 
feedback on the content of participants’ weekly assignments. This instructor 
also provided direct answers to questions participants had about the content of 
assigned readings and activities. The online facilitator played a minor role and 
was responsible for helping participants stay on pace with the course mate- 
rial and addressed technical or administrative questions raised by individual 
participants. All communications between the participants and the instruc- 
tors occurred via e-mail and were not shared with other participants. Finally, 
although participants were encouraged to complete weekly assignments, they 
were allowed to work through the course material at their own pace. 

Group 4 received no support. Participants had access to only an online 
coordinator who did not take the role of an instructor and was available only 
to answer technical questions and to provide a cursory review of assignments 
submitted by participants. There was no access to a discussion board or interac- 
tions among participants. When participants asked questions about content, the 
coordinator would direct them to sections of the assigned reading. Participants 
were allowed to work through the course at their own pace. When they submit- 
ted an assignment, the only feedback they received confirmed that the assign- 
ment was received and that it fulfilled the requirement for the course. No other 
e-mails were initiated by the coordinator except for two messages that reminded 
participants of the passage of time and, in a few cases, requested that partici- 
pants complete the data-collection instruments. 

Participants who completed the course and the data-collection instruments 
were awarded either five quarter or three semester hours of graduate course 
credit or a stipend award of $ 200 . 

Participants and Instructors 

Participation in the study was limited to middle school teachers who were 
currently teaching at least one algebra course. Messages inviting teachers to par- 
ticipate in the study were distributed via a variety of listservs. Study participants 
included 231 teachers who responded to the invitation, were currently teaching 
pre-algebra or algebra to seventh or eighth grade students, and had a working 
e-mail address. Teachers were stratified by gender and randomly assigned to one 
of the four treatment groups, with initial registration in the four conditions 
ranging from 57 to 59 teachers. We then divided each group into two cohorts, 

A and B, to have fewer teacher participants per section of the course. The start- 
ing cohort groups for each condition ranged from 28 to 30 participants. 

Three instructors and four facilitators were hired to run the sessions. Instruc- 
tors were required to be experienced middle school mathematics teachers who 
also had expertise in providing professional development in mathematics. The 
facilitators were not required to have a background in mathematics but were 
required to have experience in online facilitation. To control for the effect that 
an instructor may have on the learning outcomes of the course, each instructor 
and facilitator ran two different versions of the course. 


Journal of Research on Technology in Education 

Copyright © 2009, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int’l), iste@iste.org, www.iste.org. All rights reserved. 


451 



Table 3: Instrument List 


Instrument 

Administered 

Background survey 

Precourse 

Pedagogy survey 

Pre- and postcourse 

Math assessment 

Pre- and postcourse 

Student survey 

Pre- and postcourse 

Teacher log 

Pre- and postcourse 

Course evaluation 

Postcourse 


Data-collection Instruments 

The study presented here focused on the effect that the four versions of the 
online course had on the intended outcomes of the course. To collect data 
about the intended outcomes, six instruments were employed. These instru- 
ments included a background survey, a pedagogical beliefs and practices survey, 
a measure of teachers’ understanding of teaching algebraic concepts (referred to 
as math assessment), a student survey, a teacher log, and a course evaluation. As 
shown in Table 3, the background survey, pedagogy survey, teacher log, and the 
math assessment were administered prior to the start of the course. The peda- 
gogy survey, math assessment, student survey, teacher logs, and course evalu- 
ation were administered after the course was completed. We describe each of 
the instruments briefly below; the complete instruments are available at http:// 
www.bc.edu/research/intasc/researchprojects/optimizingOPD/OPD.shtml. 

Background survey. The background survey was designed to collect demo- 
graphic information and information about teachers’ prior experiences with 
professional development and technology use. The majority of the items on the 
background survey were closed-response. In a few instances, participants were 
requested to type in numeric values that best represented their background. 

The pedagogy survey. The pedagogy survey collected information about teach- 
ers’ pedagogical beliefs and instructional practices. All items were closed-ended 
and asked teachers to either report the frequency with which they employed 
specific instructional techniques or to indicate the degree to which they agreed 
or disagreed with a statement that focused on the value of a given instructional 
practice. We took the vast majority of items we employed for this survey from 
two previous studies that focused on the relationships between pedagogical 
beliefs and practices and the use of instructional technology in the classroom 
(Becker, 1999; Russell, O’Dwyer, Bebell, & Miranda, 2004). The specific scales 
that we formed and the associated reliabilities included (a) Teacher-Centered 
Beliefs (.76), (b) Student-Centered Beliefs (.61), (c) Instructional Use of Tech- 
nology (.55), (d) Having Students Do the Learning (.80), (e) Collecting and 
Reflecting on Student Work (.70), (f) Confidence in Teaching Math (.847), 
and (g) Confidence with Knowledge and Skills (.81). 

Math assessment. The math assessment was designed to collect information 
about teachers’ understanding of teaching the mathematical concepts covered 
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in the professional development course. The assessment required teachers to 
apply their understanding of a concept as they analyzed samples of student 
work and answered questions about what the student appeared to understand, 
what the student appeared to struggle with, and the actions the teacher would 
take to help advance the student’s understanding. We administered the assess- 
ment twice — once in the first week and once in the final week. The assessment 
presented teachers with a sample of student work for a given problem related 
to patterns or functions. It then asked the teacher to respond to a series of 
questions about the student work. For each sample, teachers were asked to (a) 
identify the content and process goals measured by the task presented to the 
student, (b) assess whether the student appears to understand the mathemat- 
ics required for the problem, (c) determine whether any misconceptions were 
present in the student work, (d) specify what they would like to know about the 
student’s understanding based on the work sample, and (e) describe what ques- 
tions or problems they would use to gain further insight into the student’s un- 
derstanding. Using a scoring guide that was shown to yield reliable scores (exact 
agreement exceeded 85% for all items), two trained readers with experience in 
teaching mathematics independently scored teacher responses. When discrep- 
ancies occurred, the readers discussed their scores before reaching a consensus 
score. The scoring guide employed the following 4-point scale: 

1 . does not meet expectations 

2. partially meets expectations 

3. meets expectations 

4. exceeds expectations 

Teacher logs. We designed the teacher logs to capture information about 
teachers’ day-to-day pedagogical practices. We administered the teacher logs 
twice — once in the first week of the course and once in final week of the course. 
During each administration, teachers completed three logs, each of which 
consisted of a series of instructional strategies similar to those explored in the 
course. For each strategy, teachers were asked to indicate whether the strategy 
was (a) not used at all, (b) a minor component of the lesson, (c) a major com- 
ponent of the lesson, or (d) the most important component of the lesson. We 
then averaged the ratings provided for each activity across the logs recorded for 
each week. 

Student survey. To help triangulate data provided by teachers via the peda- 
gogy survey and the teacher logs, we asked teachers to administer a survey to 
students in their algebra classrooms. A total of 2,682 student surveys were 
received from algebra classrooms for 145 teachers for the precourse administra- 
tion, and 2,125 student surveys were received from algebra classrooms for 125 
teachers for the postcourse administration. The survey items asked students to 
indicate the frequency with which they engaged in specific learning activities 
(e.g., performing worksheets individually, working with partners to solve prob- 
lems, sharing solutions with their class, etc.) and how often their teacher em- 
ployed specific instructional strategies (e.g., asking students to explain how they 
solved a problem, showing students how to solve a problem, asking students to 
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respond to each others work, etc.)- All items were forced-choice and were used 
to form scales that represent the degree to which students reported that they en- 
gaged in student-centered activities and in teacher-directed activities. A 4-point 
scale was employed for these items: 

1 . almost always 

2. most of the time 

3. once in a while 

4. never 

Hence, a low score for a given item indicated more frequent use of the given 
instructional strategy. 

Course evaluation. We designed the course evaluation to collect information 
from teachers about the positive and negative aspects of the course. We admin- 
istered the course evaluation only as a postmeasure. 

FINDINGS 

The primary research question examined in this study asked whether the four 
conditions that differed in the types and amounts of support provided to course 
participants affected teachers’ mathematical understanding, their pedagogical 
beliefs, and their instructional practices. To this end, the majority of analyses we 
conducted for this study focused on comparing the effect of the four versions 
of the course on these three outcomes. To provide a better understanding of 
characteristics of the study participants, we first report descriptive statistics for 
several items on the background survey. 

Characteristics of Study Participants 

The majority of the participants were female (70%). Forty-eight percent indi- 
cated that they were younger than 40 years old. Twenty-nine percent of teachers 
indicated that they had been teaching for fewer than 5 years, 26% had taught 
for 5-10 years, 12% taught 10-15 years, and 29% taught for more than 15 
years. Twenty-four percent of the participants had a college major in mathemat- 
ics, 13% minored in mathematics, and 59% did not have undergraduate focus 
in mathematics. Sixty percent of participants had obtained a master’s degree. 
Finally, 34% of participants had taken an online course previously. Although 
there were subtle differences among the groups on each of these demographic 
variables, none of the differences was statistically significant. 

Completion Rate 

Of the 231 participants who agreed to participate in the study, 46% dropped 
out of the course prior to completing all assignments and the required data-col- 
lection instruments. Analysis of those who dropped out of the study indicates that 
a larger percentage of participants dropped out of the highly supported group 
(53%) than the facilitated peer support (41%), instructor support only (45%), or 
no support (44%) groups. These percentages of participants dropping out of the 
course, however, did not differ significantly among the four conditions. 
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Table 4: Number of Discussion Board Messages per Session and Other 

Discussion 


Discussion 

Highly 
supported A 

Highly 
supported B 

Facilitated peer 
support A 

Facilitated peer 
support B 

Introductions 

75 

105 

122 

82 

Session 1 

63 

88 

82 

58 

Session 2 

65 

90 

89 

85 

Session 3 

53 

106 

71 

104 

Session 4 

76 

83 

82 

100 

Session 5 

52 

56 

51 

72 

Session 6 

50 

55 

59 

76 

Session 7 

37 

54 

56 

63 

Session 8 

42 

41 

46 

72 

Other 

31 

48 

19 

46 

Total 

544 

726 

677 

758 

Participants (n) 

12 

16 

17 

17 


The characteristics of those who dropped out also did not differ among the 
four treatment groups. The vast majority of dropouts were between the ages of 
35 and 50. In addition, teachers who reported that they minored or majored in 
mathematics, were certified to teach mathematics, or were frequent computer 
users were more likely to complete the course. Follow-up surveys with the drop- 
outs indicated that a large percentage were unable to complete the course due 
to personal (divorce, moving, death in the family) or health issues within their 
families involving themselves, their children, or their parents. A smaller number 
of dropouts indicated that they found the time requirements of the course too 
demanding on top of the time required by their teaching and family obliga- 
tions. A very small percentage indicated that persistent problems with technol- 
ogy made it too difficult for them to complete the course. Similar high dropout 
rates have been reported in other studies on online courses for educators, and 
Diaz (2002) discusses the reasons this might be the case, including participants 
experiencing technical problems, course content not matching perceived needs, 
and limited time due to other professional and family demands. 

Use of Discussion Boards for Peer Interactions 
In the two conditions that had a discussion board available for peer interac- 
tions, results show that the participants used them often. Table 4 displays the 
number of messages posted within each cohort for each session of the course. 
Throughout the course, the average number of messages per person for each 
cohort was: 38.9, 40.3, 37.6, and 44.6. An informal analysis showed that the 
large majority of the messages were directly relevant to the course content. 
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Table 5: Comparison of Pedagogical Beliefs and Teacher Confidence Scale 

Scores 



Pre score 

Post score 

Post-pre 

p value 

Confidence teaching mathematics 

-0.21 

0.31 

0.52 

<.01 

Confidence in mathematics knowledge 
and skills 

-0.29 

0.57 

0.86 

<.01 

Teacher-centered 

0.19 

-0.28 

-0.47 

<.01 

Student-centered 

-0.22 

0.31 

0.53 

<.01 

Having students “do” the learning 

-0.25 

0.27 

0.52 

<.01 

Technology use 

-0.03 

0.05 

0.08 

0.37 

Collection of student work 

-0.04 

-0.03 

0.01 

0.88 


Quality of the Online Course 

Overall, participants rated the course to be of high quality. All groups report- 
ed that the assigned readings were valuable, that interactions with facilitators 
were helpful, and that they learned more from the course than they did from 
university and college courses they had taken previously. For nearly all items 
on the survey that asked about specific aspects of the course, mean ratings fell 
between valuable and very valuable. 

To examine whether participants in each of the four versions had similar 
views about the course, we conducted an ANOVA for each item on the end-of- 
course evaluation. Adjusting for multiple comparisons, we found a statistically 
significant difference for two items. The fist item asked, “How valuable was 
reflecting in learning logs in the course?” Post-hoc comparisons revealed that 
participants in the Instructor Support Online condition indicated that the logs 
were more valuable than did those teachers in the groups that had access to peer 
interactions via a discussion board. Given that teachers in the peer interactions 
groups were encouraged to record and discuss their thinking in the discussion 
board, it is logical that they would then find the learning log, in which they re- 
corded similar thoughts, less valuable than did teachers who did not have access 
to a discussion board. 

The second item asked, “How valuable was interaction with facilitators in 
the course?” Post-hoc comparisons showed that participants in the No Sup- 
port condition rated the value of interactions with facilitators lower than did 
teachers in all other groups. Given that the coordinator for this condition was 
instructed not to have extended interactions with participants, did not have 
content knowledge expertise, and provided only short comments stating that a 
submitted assignment met the requirements for the course, it is not surprising 
that teachers in the No Support condition rated the value of interactions with 
the facilitator significantly lower than did teachers in the three other groups. 

Changes in pedagogical beliefs. We designed the pedagogy survey to collect 
information about teachers’ pedagogical beliefs and practices. We administered 
the survey pre- and postcourse, and used the data to examine changes in teach- 
ers’ beliefs and practices that followed their participation in the course. Each of 
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the seven scales was standardized to have a mean of 0 and a standard deviation 
of 1 . A comparison of precourse mean scores for each scale indicated that there 
were no significant differences among groups prior to the start of the course. 
Table 5 displays the mean scale scores for all participants for the pre- and 
postcourse administrations. The column labeled Post-pre displays the change in 
mean score between the precourse and postcourse administrations. As seen in 
Table 5, after adjusting for multiple comparisons, there were significant changes 
in teachers’ responses between the pre- and postcourse administration for five of 
the seven scales (effect sizes for significant changes ranged from .47 to .86). Spe- 
cifically, teachers’ confidence in teaching mathematics, student-centered beliefs, 
belief that students should “do” the learning, and confidence in their mathemat- 
ics knowledge and skills all experienced significant increases. The value placed 
on teacher-centered instructional practices experienced a significant decrease. 

To examine whether changes in scale scores differed among the four groups, 
an AN OVA with post-hoc comparisons was conducted. No statistically signifi- 
cant differences were found among conditions in the amount of change for any 
of the pedagogical belief scales. That is, the increases appear to be comparable 
across conditions. 

Changes in instructional practices. Teachers completed three teacher logs 
prior to the course and three logs following the completion of the course. As de- 
scribed above, the logs asked teachers to indicate the extent to which each type 
of activity was a component of the logged lesson. A comparison of precourse 
responses indicated that there were no statistically significant differences among 
groups prior to the course. Of the 23 instructional activities or behaviors in- 
cluded in the log, 10 items saw significant changes in the reported importance 
prior to and following the professional development course. As seen in Table 
6, following the course, teachers reported decreases in having students work on 
or review homework during class, demonstrating concepts to the whole class, 
and addressing routine or textbook-based problems. Teachers reported increases 
in asking students to make conjectures, presenting an answer in words, asking 
follow-up questions to student responses, having students work in pairs or small 
groups, having students respond to each other, having students debate ideas or 
explain their reasoning, and having students work together on more extended 
complex problems. These changes are consistent with the intended effects the 
course was designed to have on instructional practices, suggesting that the 
course was effective in influencing teachers’ practices. 

To examine whether changes in practices differed significantly among the 
four versions of the course, we conducted an ANOVA for each item. There was 
a significant difference among groups for only one item. This item focused on 
the extent to which teachers reported leading whole-class discussions (but not 
presenting information to the whole class). For this item teachers in the No 
Support condition reported very little increase in this practice, whereas teachers 
in the three other conditions reported moderate increases. When we made ad- 
justments for multiple comparisons, the difference among groups for this item 
was no longer statistically significant. Thus, based on teachers’ instructional 
logs, all four versions of the course had similar effects on teachers’ practices. 
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Table 6: Significant Changes in Instructional Log Activities and Behavior 
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Student survey results. Prior to and following the professional development 
course, teachers administered a survey to their students that focused on in- 
structional and learning activities students engage in during class. Table 7 (page 
460) presents the items for which statistically significant changes (adjusting for 
multiple comparisons) in student responses occurred, whereas those items for 
which changes did not occur can be seen on the student survey available at the 
URL referenced above. Of 16 items presented to students, significant changes 
in student responses between the pre- and postcourse administrations occurred 
for six items. Two of these items — copying notes from the board and practicing 
problems similar to those the teacher shows students how to solve — saw sig- 
nificant decreases in reported use. Students reported increases in being asked to 
explain how they got their answers to problems, in working on math problems 
with classmates in class, in their teacher trying to understand a student’s ap- 
proach to a math problem, and in working on an extended math problem dur- 
ing class. All of these changes are consistent with those reported in the teacher 
logs and with the intended effects of the course on teacher’s instructional 
practices. As students were unaware of the content of the course in which their 
teacher participated, the converging results in the student survey data provide 
confirmatory evidence regarding the accuracy of the teachers’ self-reports about 
change in their practices. 

To examine whether changes in the instructional practices and activities 
that students reported experiencing or engaging in differed among groups, we 
conducted an AN OVA for each of the 16 student survey items. No significant 
differences were found among groups for any of the student survey items. 

This suggests that the changes in instructional practices and learning activities 
reported by students were similar across all four versions of the course. 

Math assessment. To examine the effect that participation in the online profes- 
sional development course had on teachers’ knowledge of teaching mathemat- 
ics, teachers completed a pre- and postcourse math assessment that required 
them to analyze three samples of student work. Table 8 (page 461) displays 
the mean standardized total scores awarded across all teachers for the pre- and 
postcourse test administration. A comparison of precourse scores indicate 
that there was a statistically significant differences among groups for the set of 
items that focused on Patterns, with Group 3 performing better than the other 
groups prior to the course. There was not a statistically significant difference 
among groups for the Function item set. It is important to note that the Pattern 
item set is the only measure for which a statistically significant difference was 
detected among groups prior to the course across all measures collected for this 
study. 

As seen in Table 8, scores increased on both tests for all teachers. With the 
exception of the function test score for Group 2, mean test scores also increased 
within each group on both tests. Only two score increases, however, were statis- 
tically significant, the mean Patterns test score for all participants (effect size = 
.46) and the mean Patterns test score for Group 2 (effect size = .78). 

To examine whether changes in test scores varied among groups, we conduct- 
ed an ANOVA for both the Pattern and Function test scores. No statistically 
significant differences in test score changes occurred among the four groups. 


Journal of Research on Technology in Education 

Copyright © 2009, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int’l), iste@iste.org, www.iste.org. All rights reserved. 


439 



Table 7: Statistically Significant Changes in Student Survey Items 




Pre 


Post 


Difference 


Item 

Student Survey 

N 

Mean 

N 

Mean 

N 

Post- 

P 


Question 






pre 


7 

My teacher asks 
me to explain how 
I got my answers 
to math problems. 

145 

3.19 

125 

3.27 

93 

0.10 

0.001 

9 

I work on math 
problems during 
class time with 
other students in 
my class. 

145 

2.74 

125 

2.81 

93 

0.10 

0.007 

10 

My teacher tries 
to understand my 
way of doing math 
problems. 

145 

3.00 

125 

3.12 

93 

0.08 

0.006 

11 

We copy notes 
from the board. 

145 

3.05 

125 

2.89 

93 

-0.15 

0.001 

16 

In math class, we 
work on one big 
math problem for 
a long time. 

145 

2.14 

125 

2.22 

93 

0.12 

0.001 

17 

My teacher shows 
us how to solve 
math problems 
and then we 
practice similar 
problems. 

145 

3.50 

125 

3.40 

93 

-0.13 

<0.001 


* p value for dependent t test = whether the difference between post and pre scores is significantly 
different from zero. 

DISCUSSION AND SUMMARY 


The study presented here examined the effect that different levels and types 
of supports for an online professional development course had on the intended 
learning outcomes. In the Highly Supported condition, participants worked 
through course readings and assignments as a group, were required to interact 
with their colleagues via a discussion board, and received support from both a 
content expert instructor and an online learning facilitator. In the Facilitated 
Peer Support condition, participants worked through the course as a group, 
interacting with each other via a discussion board, with process guidance from 
an online facilitator who did not have expertise in mathematics education 
instruction. In the Instructor Support Only condition, participants worked 
through the course material at their own pace and were not able to interact with 
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Table 8: Mean Score Change for Mathematics Understanding Test 




Pre 

Post 




Treatment Group 

Mean 

N 

Mean 

N 

Post-Pre 

P 


Total 

-0.30 

60 

0.17 

66 

0.46 

<0.01 


1. Highly 








supported 

-0.43 

12 

-0.03 

16 

0.39 

0.30 


2. Facilitated peer 







Pattern 

support 

-0.66 

17 

0.13 

17 

0.78 

0.01 


3. Instructor 








support only 

0.28 

15 

0.31 

17 

0.03 

0.94 


4. No support 

-0.35 

16 

0.27 

16 

0.62 

0.09 


Total 

0.01 

66 

0.14 

60 

0.13 

0.41 


1. Highly 








supported 

0.26 

16 

0.03 

12 

-0.24 

0.43 


2. Facilitated peer 







Function 

support 

-0.13 

17 

-0.04 

17 

0.09 

0.78 


3. Instructor 








support only 

0.00 

17 

0.32 

15 

0.32 

0.94 


4. No support 

-0.08 

16 

0.26 

16 

0.34 

0.39 


other participants, although they did have interactions with both a mathematics 
instructor and an online facilitator. Finally, in the No Support condition, par- 
ticipants worked through the course at their own pace, were not able to interact 
with other participants, and had minimal interactions with a coordinator who 
did not have content expertise. 

Overall, the course yielded positive effects on the intended outcomes across 
all four conditions. On average, teachers’ pedagogical beliefs shifted to become 
less teacher directed and more student centered. Teachers strengthened their 
belief that students could do their own learning and became more confident 
in their mathematical skills and ability to teach mathematics. As reported by 
both teachers and their students, teachers’ instructional practices also shifted 
in intended ways. On average, teachers decreased the use of worksheets and 
shorter problems. In addition, teachers increased the frequency with which 
they asked students to explain their thinking, write about mathematics, work 
with peers on extended problems, and discuss strategies for solving mathemati- 
cal problems. Teachers also experienced some increases, albeit less pronounced, 
in their knowledge of mathematics teaching. On average, teachers’ scores on 
the mathematics Pattern test increased significantly. Increases on the Function 
test, however, were not statistically significant. Overall, the results show that 
the 8-week online course was successful in enabling participants to achieve the 
defined learning goals, as was found in the prior study. 

However, we also found that the unexpected outcome from the prior study 
was replicated, even with the addition of a stronger contrast between the Highly 
Supported and No Support conditions. As in the prior study, the positive effects 

Journal of Research on Technology in Education 46 1 

Copyright © 2009, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int’l), iste@iste.org, www.iste.org. All rights reserved. 



of the course did not vary across support conditions. For the mathematics test, 
the pedagogical beliefs survey, the student survey, and the instructional logs, we 
found no significant differences among the four versions of the course. 

In addition, despite the dramatic difference in the supports available across 
conditions, teachers’ ratings were generally consistent across the four versions. 
The two items for which responses differed significantly are easily explained: (a) 
Teachers who had peer interactions available found the individual learning logs 
less useful than those who did not have peer interactions available, as the use 
of these logs was somewhat redundant with the peer exchanges via the discus- 
sion board, and (b) Teachers in the No Support condition found interactions 
with the facilitator less valuable than teachers in all other versions of the course, 
as the types of interactions in which the facilitator engaged were intentionally 
limited in this condition. 

The similarity of effects across the four versions is surprising given the em- 
phasis in the literature on the importance of interactions among participants 
in online courses. Although substantial interaction occurred in the two high- 
interaction versions of the course, absolutely no interaction among participants 
occurred in the two low-interaction versions of the course. Yet the outcomes 
were nearly identical. Similarly, the literature emphasizes the importance of 
facilitation, but our results showed no differences in learning outcomes between 
conditions that were highly facilitated by both a content expert and an online 
learning facilitator and those that lacked one or both of these types of support. 
Given the large differences in the level of interactions among participants and 
the level of facilitation, these findings are particularly surprising. Moreover, the 
findings suggest that when a course is well designed and employs high-quality 
reading material and learning activities, the high levels of facilitation or inter- 
actions among participants may not be necessary to produce positive effects 
on teachers’ pedagogical beliefs, instructional practices, and, to a lesser extent, 
understanding of mathematics teaching. 

Although these findings replicate those from a prior study, they must be 
placed in the context of the study’s limitations. First, the study focused on only 
one course delivered only to middle school mathematics teachers. Had a differ- 
ent course that focused on different types of teachers or different content been 
employed, the outcomes may have been different. 

Second, all conditions of the course experienced considerable attrition. On 
average, 46% of the teachers who began a course did not finish the course or 
the required data-collection instruments. The fact that nearly half of the teach- 
ers did not complete the course may mean that the course either did not meet 
their needs or was too challenging for them. Although the characteristics of 
teachers who dropped out of the course did not differ among the four condi- 
tions, it is plausible that had these teachers persisted, different findings may 
have resulted. 

Third, the course employed for this study was well designed and employed 
high-quality learning materials and activities. In addition, the course lasted 
for 8 weeks. For a course that is shorter in length or that employs materials of 
a lower quality, interactions among participants and the amount and type of 
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facilitation may provide valuable supplemental opportunities for learning. Thus, 
had this study employed a shorter course or a course that employed lower-quali- 
ty materials and activities, different findings may have resulted. 

Fourth, it is possible that the data-collection instruments were not sensitive 
enough to detect subtle differences in changes that occurred among groups in 
participants’ beliefs, practices, and understanding. However, it is important to 
note that the instruments were sensitive enough to detect changes that occurred 
between the period when the instruments were completed prior to the course 
and following the course. Given that these changes were detected, it seems 
unlikely that a lack of instrument sensitivity is a plausible explanation for why 
changes among groups were not detected. 

Finally, all of the participants in this course were volunteers who were likely 
to be highly motivated individuals who were sincerely interested in developing 
their algebraic teaching skills. In many cases, however, participation in pro- 
fessional development is required by a school or district. In such cases, some 
teachers may be less motivated and engaged in the learning. If presented with a 
self-paced version of the course, some of these teachers might be unmotivated 
to complete the reading and activities and may make a minimal effort when 
completing assignments. The resulting effects of the course might then be 
smaller. 

Despite these limitations, this study provides preliminary evidence that a 
well-designed online course designed as a self-study with minimal interactions 
may be as effective as one that is highly interactive with both instructors and 
peer participants. Future studies may wish to explore this issue using different 
courses, content areas, and methods of recruiting teachers. Specifically, rather 
than recruiting teachers by asking them to volunteer to participate in the study, 
the potential effect of motivation might be limited by recruiting multiple school 
systems that want to implement professional development for all teachers and 
then manipulate conditions across teachers who are required to participate in 
the program. In addition, it would be informative to examine how the effects 
on the intended learning outcomes found in this study compare to the effects 
that occur in a face-to-face delivery format. It is clear, however, that an OPD 
course can have very positive effects on teachers’ knowledge, pedagogical beliefs, 
and instructional practices, and that the magnitude of these effects may be 
invariant across different levels of facilitation and interactions when a course is 
well designed and employs high-quality learning materials. 

Acknowledgments 

The authors wish to thank their colleagues, Heidi Larson, Josephine Louie, 
and Amy Goldberg, for their contributions to this work. This material is based 
upon work supported by the National Science Foundation under Grant No. 
0309401 to Education Development Center Inc. (EDC). Any opinions, find- 
ings, and conclusions or recommendations expressed in this material are those 
of the author and do not necessarily reflect the views of the National Science 
Foundation. 

Journal of Research on Technology in Education 463 

Copyright © 2009, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int’l), iste@iste.org, www.iste.org. All rights reserved. 



Contributors 

Michael Russell is an associate professor at Boston College’s Lynch School of 
Education and Director of the Technology and Assessment Study Collabora- 
tive. (Address: 332 Campion Hall, Boston College, Chestnut Hill, MA 02467; 
E-mail: russelmh@bc.edu) 

Glenn Kleiman is executive director of the Friday Institute for Educational 
Innovation and professor of educational leadership and policy studies at the 
North Carolina State University College of Education. He was Vice President 
and Center Director at Education Development Center, Inc. (EDC) while the 
research reported in this article was conducted. (Address: 1890 Main Campus 
Drive, Raleigh NC 27606; Phone: +1.919.513.8509; E-mail: glenn_kleiman@ 
ncsu.edu) 

Rebecca Carey is a project director at the Education Development Center and 
is currently working for the Regional Education Laboratory for the Northeast 
and Islands. (Address: EDC, 55 Chapel Street, Newton, MA 02458; Phone: 

+ 1.617.618.2892; E-mail: rcarey@edc.org) 

Joanne Douglas is a research associate at the Technology and Assessment 
Study Collaborative at Boston College. (Address: 332 Campion Hall, Boston 
College, Chestnut Hill, MA 02467; E-mail: jldouglas31@hotmail.com) 

References 

Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing 
teaching presence in a computer conferencing context. Journal of Asynchronous 
Learning Networks, 5(2), 1-17. 

Ball, D. L., & Cohen, D. (1999). Developing practice, developing practitio- 
ners: Toward a practice-based theory of professional education. In L. Darling- 
Hammond & L. Sykes (eds.), Teaching as the Learning Profession (pp. 3-32). 
San Francisco: Jossey-Bass. 

Becker, H. (1999). Internet use by teachers: Conditions of professional use and 
teacher-directed student use. Irvine, CA: Center for Research on Information 
Technology and Organizations. Retrieved December 10, 2008, from http:// 
www.crito.uci.edu/tlc/findings/internet-use/startpage.htm 
Borasi, R., & Fonzi, J. (2002). Professional development that supports school 
mathematics reform. Retrieved November 25, 2002, from www.nsf.gov/ 
pubs/2002/nsf02084/start.htm 

Cary, R., Kleiman, G., Russell, M., & Douglas Venable, J. (2008). Online 
courses for math teachers: Comparing self-pace and facilitated cohort ap- 
proaches. Journal of Technology, Learning and Assessment, 7(3). Retrieved 
December 26, 2008, from http://escholarship.bc.edu/jtla/vol7/3/ 
Darling-Hammond, L. & McLaughlin, M. W. (1995). Policies that support 
professional development in an era of reform. Phi Delta Kappan, 76(8), 
597-604. 

Dede, C. (Ed.). (2006). Online professional development for teachers: Emerging 
models and methods. Cambridge, MA: Harvard Education Press. 

Diaz, D. P. (2002). Online drop rates revisited. The Technology Source, May/ 
June. Retrieved December 10, 2008, from http://ts.mivu.org/default. 
asp?show=article&id= 1034 


464 


Summer 2009: Volume 41 Number 4 

Copyright © 2009, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Inti), iste@iste.org, www.iste.org. All rights reserved. 



Docherty, A., & Sandhu, H. (2006). Student-perceived barriers and facilitators 
to e-learning in continuing professional development in primary care. Educa- 
tion for Primary Care, 17, 343-353. 

Driscoll, M. (1999). Fostering algebraic thinking: A guide for teachers, Grades 

6-10. Boston: Heinemann. 

DuFour, R. (2002). Taking on loneliness. In A. Thorson (Ed.), By your own 
design: A teacher’s professional learning guide (pp. 24-25). Columbus, OH: 
Eisenhower National Clearinghouse for Mathematics and Science Education. 

Florida Instructional Technology Resource Center at UCF. (2000). Online staff 
development: Lessons learned from developers, from facilitators, from partici- 
pants. Orlando, FL: Florida Department of Education Bureau of Educational 
Technology. 

Galley, M. (2002). E-training offers options. Education Week, 21(35), 41-44. 

Garrison, R. D., & Cleveland-Innes, M. (2005). Facilitating cognitive presence 
in online learning: Interaction is not enough. American Journal of Distance 
Education, 19(3), 133-148. 

Ginsburg, A., Gray, T., & Levin, D. (2004). Online professional development for 
mathematics teachers: A strategic analysis. Washington DC: National Center for 
Technology Innovation, American Institutes for Research. 

Kleiman, G. M. (2004). Meeting the need for high quality teachers: E-learning 
solutions. A paper presented at the U.S. Department of Education Secretary’s 
No Child Left Behind Leadership Summit (July 12-13). Available online 
at http://www.ed.gov/ about/ offices/list/ os/ technology/ plan / 200 4/ site/ docu- 
ments/Kleiman-MeetingtheNeed.pdf 

Lawless, K., & Pellegrino, J. (2007). Professional development in integrating 
technology into teaching and learning: Knowns, unknowns, and ways to 
pursue better questions and answers. Review of Educational Research, 77(4), 
575-614. 

National Foundation for the Improvement of Education. (1996). Teachers take 
charge of their learning: Transforming professional development for student success. 
Washington, DC: NFIE. 

National Staff Development Council. (2001). E-learning for educators: Imple- 
menting the standards for staff development. Oxford, OH. 

Nemirovsky, R., & Galvis, A. (2004). Facilitating grounded online interactions 
in video-case-based teacher professional development. Journal of Science Edu- 
cation and Technology, 73(1), 67-79. 

Richardson, J. (2002). E-learning potential: Online staff development has great 
possibilities - and pitfalls. In A. Thorson (Ed.), By your own design: A teacher’s 
professional learning guide (pp. 12-13). Columbus, OH: Eisenhower National 
Clearinghouse for Mathematics and Science Education. 

Russell, M., O’Dwyer, L., Bebell, D., & Miranda, H. (2004). Technical Report 
for the USEIT Study. Chestnut Hill, MA: Technology and Assessment Study 
Collaborative. Available online at http://www.intasc.org/PDF/useit_rl 1. pdf 

Sargeant, J., Curran, V., Allen, M., Jarvis-Selinger, S., & Ho, K. (2006). Facili- 
tating interpersonal interaction and learning online: Linking theory and prac- 
tice. Journal of Continuing Education in the Health Professions, 26, 128-136. 


Journal of Research on Technology in Education 

Copyright © 2009, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int’l), iste@iste.org, www.iste.org. All rights reserved. 


465 



Sparks, D. (2002). Dreaming all that we might realize. In A. Thorson (Ed.), By 
your own design: A teacher’s professional learning guide (pp. 18-23). Colum- 
bus, OH: Eisenhower National Clearinghouse for Mathematics and Science 
Education. 

Sparks, D., & Hirsh, S. (1997). A new vision for staff development. Alexandria, 
VA: Association for Supervision and Curriculum Development. 

Spicer, J. (2002). Even better than face-to-face? In A. Thorson (Ed.), By your 
own design: A teacher’s professional learning guide (pp. 32-33). Columbus, 
OH: Eisenhower National Clearinghouse for Mathematics and Science 
Education. 

Stigler, J. W., & Stevenson, H. W. (1991). How Asian teachers polish each les- 
son to perfection. American Educator, 25(1), 12-21, 43-47. 

Treacy, B., Kleiman, G., & Peterson, K. (2002). Successful online professional 
development. Leading & Learning with Technology, 3(9(1), 42-47. 

Whitehouse, P. L., Breit, L. A., McCloskey, E. M., Ketelhut, D. J., & Dede, C. 
(2006). An overview of current findings from empirical research on online 
teacher professional development. In C. Dede (Ed.), Online professional de- 
velopment for teachers: Emerging models and methods (pp. 13-29). Cambridge, 
MA: Harvard Education Press. 

Yang, S. C., & Liu, S. F. (2004). Case study of online workshop for the profes- 
sional development of teachers. Computers in Human Behavior, 20, 733-761. 


466 Summer 2009: Volume 41 Number 4 

Copyright © 2009, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int’l), iste@iste.org, www.iste.org. All rights reserved. 



